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ABSTRACT 

As educators become increasingly subject to 
accountability pressures, maximizing the worth of evaluation is a 
prudent and realistic stance. This paper suggests useful and valid 
art program evaluation models: (l) Stuff lebeam ' s (1971) CIPP 
(context, input, process, and product) model; (2) Stake's Countenance 
Model (responsive evaluation); (3) McClean*s (1975) model which looks 
at total school content as context for the arts; and (4) Kushner*s 
(1987) open conversations with audiences and participants as a 
qualitative evaluation model. The staff of an arts program undergoing 
outside evaluation should: (1) clarify general goals of the program 
and articulate a philosophical vision unifying the program staff as 
iniuch as possible behind these goals; (2) take an active role in the 
selection of an evaluator; (3) clearly articulate the program to the 
evaluator; and (4) help the evaluator make an accurate translation of 
artistic concepts into the final report language. Articulation and 
accountability of the art program may result in increased recognition 
for the arts. Contains 16 references. (MM) 
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"(The arts) plainly have been among man*s most effective means 
of expression and a fundamental part of his value system for thousands 
of years, whether local or nation guidelines say so or not. Do we really 
need research to tell us that all over again?" (Morrison, 1975, pg. 74) 

Who among the community of arts educators cannot identify with the above 
statement made by the former director of the Arts in Education Program for the John 
D. Rockefeller, III fund? In this to-the-point statement. Jack Morrison articulates the 
frustration many artists feel living in an increasingly product-valuing society. The 
arts have never been fertile ground for researchers intent on demonstrating the 
"worth" of educational programs through empirical data and traditional 
quantitatively-based methods. As the demands for such justification has spread 
through the educational community, those in arts education have been among the 
most actively resistant. And to a large degree, they have been successful, at least at 
some levels. While secondary programs were increasingly subjected to 
state-mandated objectively based evaluation through the 60s and 70s, colleges and 
universities remained relatively unaffected. Writing in 1968, Henry S. Dyer 
commented: 

"The . . .concept of testing (in the arts) is valid and 
laudable; it is one that in more hortatory moods I myself 
have urged. But it is an ideal that at the moment seems so 
far from realization in practice that its impact ... is 
probably negligible." (Dyer, 1968, pg. 85) 

This insulated position enjoyed by higher education has undergone substantial 

change through the 1980s as economic and political circumstances have combined to 

create an environment where calls for accountability are increasingly heard, 
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especially at the upward end of the educational spectrum. Much of the blame 
(credit?) for this movement can be given to the increasing prevalence of managgrs 
compared to leaders in the education hierarchy; but certainly the call for a more 
detached, if not to say objective, view of the arts is nothing new, nor is it confined to 
tlie educational establishment. As far back as Kant's Critique Of JudggmgDt. a case 
was being made that objective approaches to aesthetic issues could be devised. Social 
theorist Alvin Toffler makes the point that "no system of social accounting could 
possibly be complete without indicators of the cultural life of the nation." (Toffler, 
1970. pg. 53) 

Like it or not, educators in the arts at all levels are increasingly subject to the 
environment of accountability. Whether couched in terms such as ''outcomes 
evaluation", or baldfacedly presented by an administrator as a need to "figure out 
what you're doing and if you're worth it", arts educators must learn to face some very 
challenging, unfamiliar and perhaps unwanted "music." Given these realities of 
public education entering the 1990s, three basic procedural alternatives present 
themselves: 

1. Fight evaluation initiatives. 

2. Ignore evaluation initiatives. 

3. Actively work to maximize the evaluation's worth. 

It is this paper's premise that upon examination of these choices, the third is clearly 
the most prudent and realistic. 

Laura Chapman addressed the dilemma facing the arts educator in responding 
to evaluation initiatives. She writes pointedly about the fiscal assumptions that can 
underlie many calls for evaluation. 

"Surely it is morally wrong and potentially dangerous for 'the state' to 
measure worth of human behavior and achievement in monetary terms. 
Tb.i greatest wrong of the cost accounting movement is the belief that 
purely rational decision are appropriate^ efficient and accurate. Our 
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moral and professional inlegrily demands thai we say no to (these) 
efforts." (Chapman, 1973, pg. Ill) 

Although some might ask for more restrained Slanguage, few arts educators would 

disagree with Chapman's emphasis on professional ethics in looking at the situation. 

It is important that the arts educator not forget (or ignore) the very qualities that 

make the arts so unique and valuable. The impulse to do this, to throw an artificially 

woven blanket of pseudo-objectivity over the arts, is a tempting one; many (probably 

most) administrators would prefer this if for no other reason than it is the approach 

with which they are most familiar. Braskamp and Brown warn against this course of 

action. 

"If art education is viewed (only) as an enculturation process, . 
.assessment of cognitive outcomes is generally not a problem. Indeed, it 
may be too easy." (Braskamp and Brown, 1975, pg. 68) 

The arts educator must resist such temptation, but they must also resist the other 

temptation implied by Chapman: the urge to simply dig in one's heels and summon 

all the negative energy available to fight the whole idea of evaluation, which in 

many cases is becoming inevitable in some form or other. 

By fighting the entire concept of evaluation, or worse by simply ignoring it, 

the artist/educator may be abdicating whatever influence he/she may have over the 

evaluation process, thus increasing the odds of an inaccurate or spurious evaluation 

being conducted. Braskamp and Brown are strenuous aavocates of artist involvement 

in any evaluation process. 

"If artists are convinced that an accountability plan which uses 
objective test measures and student behavioral changes will be 
inadequate and incomplete, they should be prepared to propose 
alternative plans." (Braskamp and Brown, 1975, pg. 65). 

The degree of confidence in the idea of formal evaluation may vary from passionate 

cyrJjism to resigned acceptance (perhaps even so far as a semi-enthusiastic 
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welcome?) depending on the artist/leachers involved, *>ul the argument regarding 

the need for involvement in the process on the part of the artist is a compelling one. 

"Willingness to demonstrate accountability does not necessarily mean a 

benign acceptance of a plan devised by only one party . . . The artist 

should not allow the evaluator to solely decide which criteria to assess." 
(Braskamp and Brown, 1975, pg. 65, 69) 

THE ARTS AND THE DEVELQPINn FIELD OF EDUCATIONAL EVALUATION 

If the arts are to be taken (perhaps kicking and screaming) into the era of 

accountability, its practitioners can take some comfort in the fact th'^t the field of 

educational evaluation shows signs of willingness to meet the arts halfway. As 

evaluation has distinguished itself from the field of research-based educational 

measurement, many leaders have brought forth provocative evaluation models. As 

far back as 1964, researchers Berelson and Steiner reinforced the need for such 

models with their observations regarding the arts and traditional testing procedures. 

"Among artists, the correlation between rated quality of work and 
measured intelligence is zero or slightly negative." (Berelson and 
Steiner, 1964, pg. 228-229) 

Spurred on by such findings, the 1970s were characterized by the introduction of 

new evaluation methodologies oriented toward qualitative processes rather than the 

quantitative research of measurement specialists. Not only were these methods more 

applicable to the arts, some of them seemed to demonstrate a particular afflnity with 

the artistic mindset. Elliot W. Eisner (1975) articulated the concept of "educational 

connoisseurship", comparing it to the function served by a theatre or art critic. 

Robert Stake made a similar comparison to the art critic, comparing the qualities of a 

work of art to an educational program. 

"A work of art has no single true value. A(n) (arts) program has no 
single true value." (Stake, 1975, pg. 25) 
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Given the work of these and other pioneers in qualitative evaluation, the 
looming specter of an evaluative establishment dominated by behavioral scientists 
and objective data collection began to recede. In fact, arts advocate Morrison 
reflected a growing confidence in educational evaluation, stating "The professional 
evaluator ... is more aware than anyone of the quicksilver nature of evaluating arts 
projects." (Morrison, 1975, pg. 73) 

Out of this climate came several evaluation models which should be considered 

by any arts program undergoing mandated formal evaluation. Among the best 

known of these is former music teacher Daniel Stufflebeam*s (1971) CIPP model. Its 

acronym name is derived from the four procedural steps included in the model: 

analysis of context, input, process and product. Although this model has been 

employed very effectively to facilitate management decision making in business p^d 

other quantitative areas, its flexibility allows it to serve as the basis for an effective 

evaluation of an arts program. Especially in the areas of context and process 

evaluation, the CIPP plan allows for introduction of detailed qualitative description 

that, if effectively conveyed, can result in an informed, useful evaluation. But it is 

important that an outside evaluator be made completely cognizant oi the unique 

context and processes of the arts. 

"The (evaluator) may need to be made aware of the specific role of the 
artist in education. The obligation is the artist's . . .(who) cannot afford 
to be made silent." (Braskamp and Brown, 1975, pg. 65) 

One of the foremost leaders of the effort to develop evaluation models suitable 

for the arts, Robert Stake presented the idea of "responsive evaluation" as 

"An educational evaluation . . . orient(ed) more directly to program 
activitif^s than to program intents; responding to audience 
requirements for information; (where) different value perspectives 
present are referred to in reporting the success or failure of the 
program." (Stake, 1975, pg. 14) 
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In establishing a structure for such an evaluation, Stake developed a "matrix*' system 
whereby the intents, observation, standards and judgments of a program would be 
developed in three categories chronologically labeled Antecedents (conditions 
existing prior to training). Transactions (the instruction process) and Outcomes. 
Later given the title "Countenance Model" by Stake, this system has considerable 
applicability to an artistic program, its chief advantages being its clarity and 
comprehensiveness. Use of the Countenance Model could be quite helpful in 
situations where evaluation credibility is likely to be questioned; its specific and 
somewhat technical presentation belies its flexibility and adaptability to artistic 
needs. 

One of the most intriguing approaches to the evaluation ol art programs was 
presented by Leslie D. McClean (1975). It is particularly attractive to artists in that 
she proposes that the evaluator look at the total school content; that the evaluation 
focus on how hospitable the school as a total institution is to the arts. While this 
focus may not be the favorite of administrative audiences, McClean^s is one of the few 
models to acknowledge that the arts program does not exist in a vacuum. Just as no 
artistic institution can thrive without a supportive community, no educational 
program can thrive without a supportive school environment. 

McClean presents guidelines that help an evaluator determine "observable 
features of the school which (may) reveal appreciation for, or neglect of, the arts". 
First among her four categories is the total school environment. McClean urges 
evaluators to thoroughly look through the entire school. Is art displayed (outside of 
art classrooms)? Is it numerous or scarce? What is its quality both in terms of 
technique and originality? Does it reflect vital activity of engaged student minds? 
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The McClean model outlines similar guiding questions related to arts workspace, arts 
program output, and direct school support for art programs. 

Obviously a mandated evaluation which utilized onlv this model in its 
information gathering would probably have severe credibility problems. But as a 
complementary approach to an otherwise formulaic evaluation, McClean^s ideas 
certainly have merit and would in all likelihood prove to be a valuable component of 
any arts program evaluation. 

Given the amount of activity and exploration in the field of arts evaluation 
that occurred during the 1970's, it is notable that a search of sources through the 
1980s reveals very little new literature. Guba and Lincoln (1981) have presented 
clear and articulate methods for qualitative evaluation, as did Patton (1987); but work 
specifically addressed to the arts is minimal. 

One recent essay should give heart to arts educators because of its fresh, 
unpretentious presentation and its common-sense, sensitive conclusions. British 
evaluator Saville Kushner observes that "really potent evaluations are embodied in 
conversations with audiences and participants." (Kushner, 1987, pg. 228) Although 
many artists may claim they don't need a distinguished expert to point out the value 
of conversation, Kushner's emphasis on this emergent, open-ended process of data 
collection does indicate the distance the evaluation mainstream has traveled toward 
developing methods compatible with subjective disciplines such as the arts and 
humanities. 

Kushner also raises a flag of caution that should be carefully heeded by all 
evaluators - especially those unfamiliar with the personal and emotional nature of 
work in the creative arts. The case with which the well-intentioned evaluator may 
be drawn in to the personalities and energy surrounding creative work is 
emphasized. 
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"Evaluating innovations in the arts starts with holding conversations - 
interviews that have an evaluative bite to them . . .(but) it is hard to 
hold such conversations whilst maintaining a professional distance." 
(Kushner, 1987, pg. 223) 

While Kushner describes the unique challenges of arts evaluation, the positive 
side of working with a generally gregarious group of people is also acknowledged. 
Kushner finds that "much of the evaluation is conducted by the participants, and my 
role is simply to ease conversation, edit and publish." (Kushner, 1987, pg. 220) In a 
concise statement regarding the nature of evaluating arts programs, Kushner says 
simply "This is an intimate form of research." 

Evaluating the arts is a challenging proposition. Morrison outlines a few of 

the many things an evaluator must do well. 

"Honest reporting (of artistic evaluation) takes imagination and 
insightful interpretation with great integrity. It means the use and 
respect of subjective data carefully collected and ordered. It means 
demystifying objective data in the subjective realm of feelings." 
(Morrison, 1975, pg. 75) 

Whether any mere mortal is up to such a task is debatable, and many arts educators 
are quick to agree with the evaluators who candidly admitted that "the truly objective 
judge (of the arts) is often the passage of time*. (Braskamp and Brown, 1975, pg. 59) 
But assuming that father/mother time is not available, and the local school board or 
state education department is still insisting on a formal program evaluation, the arts 
educator must be prepared to maximize the benefit (or minimize the damage) of any 
such process. 

THE ARTISTS* ROLE IN PROGRAM EVALUATION 

When faced with the prospect of a mandated outside evaluation, many artists' 
first impulse is to flee. It is of absolute importance that this urge not be given in to. 
Such evaluations are an increasingly common aspect of the trend toward overt 
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educational accountability, and despite the pleas from subjectively oriented 
disciplines, they seem to be here to stay in the foreseeable future - especially in the 
public secondary schools and increasingly in state run university systems. The artist 
must develop a set of "coping skills" to maximize the influence he/she can bring to 
bear on the process. 

Toward that end, four distinct areas of activity are recommended to the staff of 
an "^rts program undergoing outside evaluation: 

1. Clarify general goals of the program and articulate a philosophical vision, 

unifying the program staff as possible behind these goals. 

2. Take an active role in the selection of an evaluator. 

3. Clearly articulate the program to the evaluator. 

4. Help the evaluator make an accurate translation of artistic concepts into the 

final report language. 

Many artists find the idea of a "mission statement" to be trivial, aggravating or 
even odious. This reactionary smokescreen obscures the fact that these negative 
feelings are often a function of the incredible difficulty encountered in articulating 
such a statement. It is extremely difficult to devise a concise, non-jargonistic 
statement that accurately conveys even the slightest essence of such a complex and 
personal process as creativity. The difficulty of explaining the process is in itself an 
exhortation to do so, for if it is difficult for the artist to articulate the process, pity the 
conscientious outside evaluator who nobly (and in all likelihood unsuccessfully) 
struggles to capture a program's essence based on a short on-site visit. Dyer conveys 
the frustration felt by many professional evaluators who encounter ambiguity at ail 
levels of the arts education hierarchy, stating the "dimensions of (expected artistic) 
achievement are still largely undefined in the minds of professors' (Dyer, 1968, pg. 
91) 
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Clarification of general goals are a practical imperative for an effective 
evaluation, ^ut artists should also perceive the innate benefits that may accrue from 
the time taken to consolidate artistic vision. As art educator Bennett Reimer argues, 
"Arts programs need a long range ph losophical vision clearly articulated and 
generally shared". (Reimer, 1980, pg. 120) All too often in the arts, individuals 
become so caught up in the creativity or deadline pressures of artistic v^ork that 
philosophical thought becomes a luxury they feel they cannot afford. An outside 
evaluation can be used as a **stick" to encourage pursuit of this "caiiol" ihat often 
slips from view outside of the ' ffs artistic blinders. 

It should be emphasized that any mission statement should grow out of the 
thought and feelings of the artists themselves; no attempt should be place to overlay 
the particular lexicon of the professional education industry. Behavioral objectives 
subdivided into decimal groupings not only will turn program staff off, they also may 
reinforce the prejudices of an evaluator. Arts educators should realize, however, that 
if they fail to define goals on their own terms, a system such as behavioral objectives 
is more likely to be imposed on them. 

Perhaps no step in the evaluation process is more important than selection of 
an evaluator, and artists need not be a passive party in this process. In most cases, 
the first step would be to advocate a "connoisseurial" or expertise-oriented (Worthen 
and Sanders, 1987) model, with experts being nominated by the arts program staff. 
Should administration demand a more traditional approach, artists should take an 
active role in identifying evaluation professionals knowledgeable about and 
sympathetic to models such as those outlined earlier. All evaluators are not equal or 
alike and any investment of artist' time used to identify an appropriate evaluator 
will, in all likelihood, be effort extremely well "pent. 
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Once an evaluator is hired, an arts program must dedicate itself to 
communicating clearly to the evaluator its nature and goals. If clarification of goals 
has. taken place prior to the hiring of the evaluator, this should be possible; if not 
this may be a difficult and inefficiem, if not impossible, process. A special point 
should be made here: if an evaluator of low artistic credibility is forced on a 
program, the need for patient and clear program articulation becomes nmxfi 
pronounced. This is emphasized here because human native reinforces the opposite; 
we are much more inclined to talk at length with people who understand (and 
approve of) us than we are with people who are more ignorant of the facts and value 
behind our work. It is of crucial importance that this instinct be consciously 
overcome in the case of an assigned evaluator who is inappropriate or inadequate. To 
ignore such an individual, or to provide only cursory information, is to invite a 
report that is magnified in its shortcomings and inaccuracies. 

Finally, an arts program should work with the evaluator to ensure that an 
effective report is presented to the sponsoring agency/administration. This means 
that every effort should be made to ensure that accurate, fair COntSP l is in evidence. 
However, artists should check their impulse to become intrusively involved in the 
S ty} ^ of the report. Artists will naturally wish to see the report presented in artistic 
terms, but if they are successful in advocating this, an important strategic 
opportunity may be lost. An evaluation can serve as an opportunity for an outside 
expert to communicate the worth of the program to an educational 
agency/administration in the administrator's own lansuagg- Both the perceived 
credibility of the evaluator and the increased efficiency of communication allowed 
by such wriUng can be extremely useful to an arts program in advocating its cause. 
Therefore, artists should work with the evaluator to assist in the translation of 
artistic concepts into the most effective semantic style possible. This process takes 
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time and patience as well, but it can be the last piece of the puzzle whereby an 
otherwise bothersome outside evaluation can be employed as a vehicle for effective 
arts advocacy. 



It may be true that the arts are less fully understood by the educational 
establishment than many other disciplines. And it may be true that many of the 
traditional methods of program evaluation are more dubious in application to the arts 
than to many other disciplines. Nevertheless, two factors seem evident in the field of 
arts education as we begin the nineties: arts education (along with education in 
general) is being held to progressively more rigorous standards of accountability; 
and arts education is in dire need of active, effective and inspired advocacy on all 
levels, it is the arts educators' considerable challenge to bring his/her creativity to 
bear in such a way that the former can be used as a means to achieve the later. While 
it may be true that "our ability to dream goes beyond our ability to explain" 
(Heussenstamm, 1987 pg. 44), perhaps by explaining at least part of our artistic 
dream, we might be given the resources to dream more boldly in the future. 
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